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Summary 


Two  mesoscale  weather  forecasting  models — the  Mesoscale  Model  Version  5  (MM5)  and  the 
Weather  Research  and  Forecast  (WRF) — were  statistically  evaluated  over  two  different 
geographical  areas,  Utah  and  western  Texas. 

Using  the  40-km  Eta  forecast  data  as  input  data,  forecast  calculations  using  both  the  MM5  and 
the  WRF  were  carried  out  and  the  results  were  compared  with  surface  observation  data.  Both 
models  tended  to  over-forecast  temperature  and  dew  point  temperature,  although  the  correlation 
coefficients  between  forecast  and  observation  were  fairly  high. 

The  statistical  parameters  for  the  MM5  were  slightly  better  than  for  those  for  the  WRF.  For  both 
the  MM5  and  the  WRF,  statistical  parameters  for  wind  vector  components  were  inferior  to  those 
for  temperature  and  dew-point  temperature,  though  the  values  for  the  WRF  were  slightly  better 
than  the  values  for  the  MM5. 

The  influences  of  different  input  data  on  the  MM5  forecast  fields  were  studied  using  both  the 
40-km  Eta  and  the  Global  Forecast  System  (GFS).  For  all  surface  meteorological  parameters,  the 
MM5  using  the  inputs  from  the  40-km  Eta  performed  better  than  the  MM5  using  inputs  from  the 
GFS.  The  importance  of  input  data  to  mesoscale  models  should  be  further  studied. 

Also,  the  WRF  forecasting  over  the  western  Texas  produced  better  statistical  results  than  the 
forecasting  over  the  Utah.  This  was  probably  due  to  the  simpler  terrain  in  western  Texas,  as 
compared  to  Utah. 
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1.  Introduction 


With  advancements  in  both  computer  hardware  and  software  (such  as  the  availability  of  a 
parallel  computer  with  distributed  memory  and  Message  Passage  Interface),  applications  of  non¬ 
hydrostatic  meteorological  models  with  many  different  physics  options  have  become  possible  for 
operational  use.  Thus,  it  is  important  to  evaluate  how  non-hydrostatic  models  perform  over  areas 
of  complex  terrain.  Two  such  non-hydrostatic  models — the  Pennsylvania  State 
University/National  Center  for  Atmospheric  Research  (NCAR)  Mesoscale  Model  Version  5 
(MM5)  and  the  Weather  Research  and  Forecast  (WRF)  model,  are  the  focus  of  this  report. 

In  a  statistical  evaluation  study  comparing  the  MM5  and  the  Battlescale  Forecast  Model  (BFM) 
(Hemni,  2003a)  over  a  complex  Utah  terrain,  the  following  results  were  obtained: 

•  No  significant  statistical  differences  in  forecast  skills  were  found  between  the  MM5, 
domain  3  (7.5-km  grid  resolution),  and  the  MM5,  domain  4  (2.5-km  grid  resolution). 
Similarly,  the  forecast  skills  of  two  BFM  (5-  and  2.5-km  grid  resolutions)  were  also  found 
to  be  statistically  similar.  Forecast  results  did  not  significantly  improve  for  either  model 
even  when  the  grid  resolutions  were  reduced. 

•  Both  the  MM5  and  the  BFM  produced  more  accurate  forecast  results  for  surface 
temperature  than  for  dew-point  temperature  and  wind  vector  components. 

•  For  all  of  the  surface  meteorological  parameters  of  temperature,  dew-point  temperature, 
and  wind  vector,  the  MM5  demonstrated  statistically  better  forecasting  skills  than  the 
BFM. 

•  Both  the  MM5  and  the  BFM  produced  better  forecasting  statistics  during  the  period  of 
April  to  June  2002  than  during  the  period  of  January  to  March  2002. 

The  forecasting  skills  of  the  MM5’s  surface  meteorological  parameters  were  statistically 
evaluated  using  three  different  planetary  boundary  layer  (PBL)  parameterization  schemes —  the 
Medium  Range  Forecast  model  (MRF),  the  Blackadar,  and  the  Eta  PBL  (Hemni,  2003b).  The 
MM5’s  surface  temperature  forecasting  skills  were  found  to  be  similar  when  using  each  of  three 
different  schemes;  all  were  slightly  positively  biased.  However,  for  dew-point  temperature,  the 
MM5  using  the  Eta  PBL  parameterization  scheme  produced  significantly  positively  biased 
results  during  the  local  daylight  hours.  For  wind  speed  and  direction,  each  scheme  produced 
similar  statistical  results:  the  correlation  coefficient  (CC)  for  wind  speed  was  between  0.3  and 
0.5,  the  mean  absolute  difference  (AD)  was  slightly  smaller  than  2.0  m/sec,  and  the  mean  wind 
direction  difference  (MWDDF)  was  between  50°  and  60°. 

A  comparison  study  of  the  forecast  skills  of  the  MM5  between  two  domains  (with  a  15-  and 
5 -km  grid  resolution)  (Hemni,  2003b)  showed  that,  for  temperature  and  dew-point  temperature, 
the  model  domain  with  the  5-km  increment  produced  slightly  better  forecasting  results  than  the 
model  domain  with  the  15-km  increment.  Also,  for  wind  speed  and  the  direction  and  vector 
components,  u  and  v,  the  model  domain  with  the  5-km  grid  resolution  produced  better  wind 
forecasts  than  the  one  using  the  15-km  grid  resolution. 
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The  MM5  produced  statistically  better  forecasting  results  for  all  surface  meteorological 
parameters  when  it  used  initialization  and  time-dependent  lateral  boundary  data  from  the 
Aviation  Forecast  Model  (AVN)  rather  than  from  the  Navy  Operational  Global  Atmospheric 
Prediction  System  (NOGAPS).  Initialization  and  lateral  boundary  value  data  are  essential  for 
producing  reliable  forecasting  results. 

In  the  continuing  efforts  of  mesoscale  model  evaluation,  the  objectives  of  this  present  study  are 
as  follows: 

•  To  compare  the  WRF  and  MM5  models.  For  this  study,  initialization  and  time-dependent 
lateral  conditions  were  provided  by  the  forecast  data  of  the  40-km  Eta  produced  by  the 
National  Center  of  Environmental  Prediction  (NCEP). 

•  To  study  the  influences  of  different  initialization  and  lateral-boundary  condition  data  on  the 
MM5  forecast,  by  using  data  provided  by  the  40-km  Eta  and  by  the  Global  Forecasting 
System  (GFS),  using  a  1°  grid  resolution.  (Both  datasets  are  from  the  NCEP.) 

•  To  examine  the  influences  of  different  geographical  and  topographical  areas  on  statistical 
parameters,  using  the  model  domains  in  Utah  and  western  Texas. 

For  the  present  study,  a  triple -nested  MM5  was  used,  and  since  a  WRF  with  nesting  capability 
was  not  available  when  the  study  was  started,  version  1 .3  of  the  WRF  (which  does  not  have  a 
nesting  capability)  was  used. 

Section  2  briefly  describes  the  MM5  and  the  WRF.  The  model  domains,  input  data,  and 
statistical  parameters  used  in  this  study  are  mentioned  in  section  3.  Section  4  shows  the  results  of 
study,  and  section  5  presents  the  conclusions. 


2.  The  MM5  and  the  WRF 


2.1  The  MM5 

The  distributed-memory  version  of  the  MM5  was  used.  The  source  code  of  the  MM5  was 
obtained  from  the  MM5  home  page  (2001). 

The  MM5  is  based  on  non-hydrostatic  dynamics  and  features  multiple-nest  capabilities  and  many 
physics  options.  Details  of  this  modeling  system  can  be  found  in  Dudhia  (1993),  Grell  et  al. 
(1994),  and  Warner  et  al.  (1992). 

For  the  present  study,  the  following  physics  options  were  employed: 

•  PBL.  The  PBL  technique  used  was  that  of  the  MRF  model  of  the  NCEP  by  Hong  and  Pan 
(1996). 

•  Precipitation  parameterization.  This  study  applied  a  simple  treatment  of  cloud  microphysics 
based  on  Dudhia  (1989),  in  which  both  ice  and  liquid  phases  are  pennitted  for  cloud  and 
precipitation,  but  mixed  phases  are  not  permitted. 

•  Cumulus  parameterization.  For  this  aspect,  the  study  used  Grell ’s  scheme,  which  is  based 
on  rate  of  destabilization,  or  quasi-equilibrium;  features  a  simple  single-cloud  scheme  with 
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updraft  and  downdraft  fluxes;  and  uses  compensating  motion  to  determine  the  heating  and 
moistening  profile  (1994). 

•  Radiation  parameterization.  Dudhia’s  scheme,  in  which  long- wave  and  short-wave 
radiation  interact  with  the  clear  atmosphere,  clouds,  precipitation,  and  the  ground  (1989), 
was  used. 

•  Ground  temperature  scheme.  This  study  utilizes  a  multilayer,  soil  temperature  model. 

2.2  The  WRF 

The  WRF  is  a  weather  forecasting  system  with  fully  compressible,  non-hydrostatic  equations 
(unlike  the  MM5,  which  is  based  on  incompressible,  non-hydrostatic  equations).  Therefore,  the 
WRF  can  be  used  to  simulate  atmospheric  phenomena,  including  acoustic  and  gravity  waves. 
Details  of  the  WRF  are  available  on  the  WRF  home  page  (2004). 

As  with  the  MM5,  the  WRF  can  use  a  number  of  different  physics  options.  For  the  present  study, 
the  following  physics  options  were  used: 

•  PBL.  A  new  scheme,  known  as  the  Yonsei  University  PBL,  was  used.  It  has  an  explicit 
representation  of  entrainment  at  the  PBL  top,  which  is  derived  from  large  eddy  modeling. 
This  scheme  partially  corrects  the  problem  of  too  much  entrainment  in  the  early  phase  of 
PBL  growth  and  also  adds  non-local  momentum  mixing  to  provide  a  realistic  wind  profile 
in  the  PBL  (Dudhia,  2004). 

•  Precipitation  parameterization.  The  WRF  Single-Moment  3-class  was  used  (Hong,  Dudhia, 
and  Chen,  2004).  In  this  scheme,  ice  crystal  number  concentration  is  dependent  on  ice  mass 
content  rather  than  temperature,  giving  realistic  ice  crystal  concentrations  and  size  that  are 
compatible  with  the  fall  speeds. 

•  Cumulus  parameterization.  The  modified  version  of  the  Kain-Fritch  scheme  (1990,  1993) 
was  selected.  This  scheme  utilizes  a  simple  cloud  model  with  moist  updrafts,  including  the 
effects  of  detrainment,  entrainment,  and  cloud  microphysics.  A  minimum  entrainment  rate 
is  imposed  to  suppress  widespread  convection  in  marginally  unstable,  relatively  dry 
environments.  Shallow  (non-precipitating)  convection  is  allowed  for  any  updraft  that  does 
not  reach  minimum  cloud  depth  for  precipitating  clouds;  this  cloud  depth  varies  as  a 
function  of  cloud-base  temperature.  In  this  version,  the  entrainment  rate  is  allowed  to  vary 
as  a  function  of  low-level  convergence. 

•  Radiation  parameterization.  For  long- wave  radiation,  the  rapid  radiative  transfer  model 
(RRTM)  (Mlawer  et  ah,  1997)  was  used.  The  RRTM  is  a  spectral-band  scheme  that  uses 
preset  tables  to  accurately  represent  long  wave  processes  due  to  water  vapor,  ozone,  CO2  , 
and  trace  gases  (if  present)  as  well  as  accounts  for  cloud  optical  depth.  For  short-wave 
radiation,  a  scheme  based  on  Dudhia  (1989)  was  used. 

•  Ground  temperature  scheme.  The  five-layer,  soil  temperature  model  used  in  the  MM5  was 
used  for  the  WRF  as  well. 

In  June  2004,  the  WRF  version  2,  with  one-way  and  two-way  multiple  nesting  capabilities,  was 
released.  However,  for  the  present  study,  the  WRF  version  1 .3  was  used  (which  was  released  in 
2003);  this  version  does  not  possess  a  nesting  capability. 
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3.  Model  Domains,  Input  Data,  and  Statistical  Parameters 


3.1  Model  Domains 


The  MM5  triple-nested  computational  grids,  depicted  in  figure  1,  were  used  for  the  Utah  model 
domains.  All  three  computational  domains  have  a  mesh  size  of  55  by  55  grid  points  and  grid 
resolutions  of  45,  15,  and  5  km  for  domains  1,  2,  and  3,  respectively.  For  the  present  study,  only 
the  forecast  fields  for  domain  3  are  compared  with  the  observed  data. 


Figure  1.  Geographical  extents  of  the  three  MM5  computational  domains  in  Utah. 
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Figure  2  shows  the  terrain  contours  of  the  MM5,  covering  domains  2  and  3.  The  locations  of  the 
GFS  grid  points  are  marked  using  the  “#”  symbol  and  the  40-km  ETA  grid  points  are  marked 
using  the  symbol. 


Figure  2.  Terrain  contours  of  the  MM5  domains  2  and  3  for  Utah. 
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The  WRF  used  the  model  domain  shown  in  figure  3.  The  domain  was  centered  at  lat.  40.5°N  and 
long.  1 12.5°W,  with  a  grid  point  number  of  52  by  52  and  a  grid  resolution  of  5  km.  In  figure  3, 
the  “*”  symbol  represents  the  location  of  the  40-km  Eta  grid  point  and  the  letter  “S”  represents 
the  location  of  the  surface  observation  sites  used  for  the  comparison  study.  The  size  and 
geographical  location  of  this  model  domain  are  similar  to  that  of  the  MM5,  domain  3,  as  shown 
in  figure  1 . 


Figure  3.  Terrain  contours  of  the  WRF  model  domain  in  Utah,  using  a  52-by-52  grid 
and  a  5-km  grid  resolution. 
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The  model  domain  of  the  western  Texas  area  for  the  WRF  is  shown  in  figure  4.  As  can  be  seen, 
this  domain  is  flatter  and  simpler  than  the  terrain  in  the  Utah  domain.  In  figure  4,  the  symbol 
represents  the  locations  of  the  404cm  Eta  grid  point  and  the  letter  “S”  represents  the  location  of 
the  Texas  Tech  University  (TTU)  Mesonet  observation  site.  The  domain  is  centered  at  33.82°N 
and  101.82°W  and  has  52  by  52  grid  points  and  a  5-km  grid  resolution. 


Figure  4.  Terrain  contours  of  the  western  Texas  model  domain  for  the  WRF. 

NOTE:  The  “*”  symbol  represents  the  40-km  Eta  grid  points;  the  letter  “S”  represents 
the  Texas  Tech.  University  Mesonet  observation  sites. 


3.2  Input  Data 

The  40-km  Eta  forecast  data  is  used  for  initialization  and  time-dependent  lateral  condition.  This 
dataset  was  used  for  the  comparison  study  of  the  MM5  and  the  WRF.  The  data  is  freely  available 
from  the  NCEP  FTP  site:  ftpprd.ncep.gov. 

GFS  data,  with  a  1°  grid  resolution,  was  also  used  to  examine  the  influences  of  different  input 
data  on  the  MM5  forecasting  over  the  Utah  domain.  This  dataset  can  also  be  obtained  from  the 
NCEP  FTP  site. 
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Both  the  MM5  and  the  WRF  were  initialized  at  1800  universal  time  coordinated  (UTC)  and  were 
run  for  30  hours.  Comparisons  between  model  output  data  and  observation  data  were  made  every 
hour,  starting  from  0000  UTC. 

Surface  observation  data  for  the  Utah  area  is  freely  available  from  the  University  of  Utah 
MesoWest  Cooperative.  A  MesoWest  data  file,  total .  dat,  contains  data,  in  15-min  intervals, 
for  a  24-h  period  for  the  western  United  States. 

Data  for  the  western  Texas  domain  was  obtained  from  the  TTU  FTP  sites,  which  contain 
monthly  files  for  each  station. 

A  daily  data  file,  containing  data  only  for  the  even  hours  of  a  24-h  period  (starting  at  0000  UTC), 
was  created  from  the  above  datasets  for  each  domain  area. 

3.3  Statistical  Parameters 

The  following  statistical  parameters  were  calculated  for  comparisons  between  the  forecast  and 
observation  data: 

•  Mean  difference  (MD) 

•  Mean  AD 

•  Root  mean  square  error  (RMSE) 

•  Root  mean  square  vector  error  (RMSVE) 

•  Correlation  coefficient  (CC) 

•  Mean  wind  direction  difference  (MWDDF) 

The  details  of  these  parameters  are  described  in  Hemni  (2003a). 


4.  Results 


A  comparison  study  of  the  MM5  and  the  WRF  over  the  Utah  model  domains  was  done  using  29 
daily  forecast  and  surface  observation  conducted  between  March  30  and  April  27,  2004. 

The  40-km  Eta  forecast  data  was  used  for  this  study.  Forecast  calculations  of  the  WRF  over  the 
western  Texas  domain  were  also  conducted  using  the  40-km  Eta  forecast  data  during  the  same 
time  period. 

To  study  the  influences  of  different  initialization  and  lateral  boundary  condition  data  (the  40-km 
Eta  versus  the  GFS),  a  15-day  period  between  April  16  through  April  30,  2004  was  selected,  and 
the  forecast  data  of  the  MM5,  domain  3,  was  compared  with  the  surface  observation  data. 

4.1  Comparison  study  of  the  MM5  and  WRF 

4.1.1  Statistical  Parameters  for  the  Entire  Dataset 

Statistical  parameters  between  forecast  data  and  surface  observation  data  were  calculated 
covering  the  entire  datasets  for  29  days,  between  March  30  2004  and  April  27  2004.  The 
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statistical  parameters  for  the  MM5  and  WRF  (given  in  tables  1  and  2,  respectively)  were  based 
on  about  24,000  data  pairs. 


Table  1.  Statistical  parameters  between  the  MM5,  domain  3,  forecast  data  and  surface  observed  data  for  the 
period  of  March  30  through  April  27,  2004. 


MD 

AD 

RMS  I 

CC 

Temperature  (°C) 

1.0 

2.5 

3.2 

0.87 

Dew-point  temperature  (°C) 

-0.2 

3.2 

4.2 

0.63 

Wind  speed  (m/sec) 

1.0 

2.2 

2.8 

0.35 

Wind  vector  x-component  (m/sec) 

-0.1 

1.9 

2.9 

0.45 

Wind  vector  y-component  (m/sec) 

0.9 

2.3 

3.1 

0.55 

NOTE:  RMSVE  =  4.2  (m/sec),  MWDDF  =  46° 

Table  2.  Statistical  parameters  between  the  WRF  forecast  data  and  surface  observed  data  for  the  period 
of  March  30  through  April  27,  2004. 


MD 

AD 

RMSE 

CC 

Temperature  (°C) 

0.7 

2.9 

3.8 

0.81 

Dew  -point  temperature  (°C) 

2.8 

3.7 

5.1 

0.64 

Wind  speed  (m/sec) 

0.5 

1.8 

2.4 

0.41 

Wind  vector  x-component  (m/sec) 

-0.2 

1.8 

2.7 

0.44 

Wind  vector  y-component  (m/sec) 

0.4 

2.0 

2.8 

0.54 

NOTE:  RMSVE  =  3.8  m/sec,  MWDDF  =  47° 
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The  scatter  diagrams  of  temperature  between  the  forecast  data  and  observation  for  the  MM5  and 
the  WRF  are  shown  in  figures  5  and  6,  respectively. 


Figure  5.  Scatter  diagrams  of  temperature  between 

forecast  data  and  observation,  for  the  MM5. 


Figure  6.  Same  as  figure  5,  except  for  the  WRF. 
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Scatter  diagrams  of  dew-point  temperature  between  forecast  and  observation  data  for  the  MM5 
and  the  WRF  are  given  in  figures  7  and  8,  respectively. 


Figure  7.  Scatter  diagram  of  dew-point  temperature  between 
forecast  and  observation  for  the  MM5. 
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Figure  8.  Same  as  figure  7,  except  for  the  WRF. 
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Scatter  diagrams  for  wind  speed  between  forecast  and  observation  data  for  the  MM5  and  the 
WRF  are  shown  in  figure  9  and  10,  respectively. 
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Figure  9.  Scatter  diagram  of  wind  speed  between  forecast  and 
observation,  for  the  MM5. 
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Figure  10.  Same  as  figure  9,  except  for  the  WRF. 
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4.1.2  Time  Series  of  the  Statistical  Parameters  (MD,  CC,  and  AD) 

The  statistical  parameters,  MD,  CC,  and  AD,  (as  mentioned  in  section  3.3)  are  calculated  at  each 
forecast  hour  using  all  available  data  for  the  29-day  period.  In  the  following  figures  11-14,  time 
series  of  the  MD,  CC,  and  AD  for  the  MM5  and  the  WRF  are  plotted  using  different  colors  (pink 
for  the  MM5  and  black  for  the  WRF). 


Figure  11.  Time  series  of  MD,  CC,  and  AD  for  temperature. 


Figure  12.  Same  as  figure  11,  except  for  dew-point  temperature. 


14 


Figure  13.  Same  as  figure  1 1,  except  for  wind  speed. 


RMSVE ( M/S )  &  MWDDF (DEG. ) 


Figure  14.  Time  series  of  RMSVE  and  MWDDF. 

From  tables  1  and  2  and  figures  5  through  14,  the  following  can  be  inferred: 

1 .  For  temperature  forecast,  the  MM5  produced  better  statistical  results,  although  both  models 
simulated  temperature  well,  producing  a  CC  greater  than  0.8  and  an  AD  between  2  °C  and 
3  °C. 

2.  As  can  be  seen  from  the  MD  values,  the  WRF  tended  to  over-forecast  dew-point 
temperature  significantly,  yielding  an  MD  of  2.8  for  the  entire  dataset,  in  contrast  to  the 
MD  of  -0.17  for  the  MM5  (see  tabs.  1  and  3). 
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3.  For  wind  speed,  the  MM5  showed  a  tendency  to  produce  greater  values  than  the  observed 
values  throughout  the  24-h  period  (see  fig.  13).  As  seen  in  both  models,  the  statistical 
parameters  relating  to  wind  vector  indicate  that  forecasting  surface  wind  by  mesoscale 
models  is  difficult  and  challenging  over  such  complex  terrains  as  seen  in  the  Utah  domain. 

4.2  Study  of  Different  Input  Data  on  the  MM5  Forecast 

In  a  previous  study  on  the  influences  of  different  global  forecast  models  (AVN  and  NOGAPS)  as 
input  data  to  the  MM5  (Henmi,  2003  b),  it  was  shown  that  an  MM5  using  the  AVN  data  as 
initialization  and  boundary  value  data  produces,  for  statistical  parameters  (MD,  CC,  RMSE,  and 
AD),  better  forecasting  results  of  surface  meteorological  parameters  than  an  MM5  using 
NOGAPS  data.  The  time  series  lines  of  statistical  parameters  for  both  AVN  and  NOGAPS  are 
almost  parallel  throughout  the  24-h  forecast  period.  This  was  especially  significant  for 
temperature  and  dew-point  temperature,  indicating  the  differences  between  AVN  and  NOGAPS 
data  were  systematic,  but  not  random,  over  the  model  domain  of  Utah. 

This  study  looked  at  the  influences  of  the  GFS  and  the  40-km  Eta  model  as  input  data  to  the 
MM5.  The  locations  of  grid  points  of  both  models  over  the  MM5  domain  of  Utah  are  shown  in 
figure  2.  In  the  MM5,  domain  3,  there  are  6  GFS  grid  points  in  contrast  to  39  Eta  grid  points. 

Table  3  shows  the  statistical  parameters  between  the  MM5  forecast  and  the  surface  observation 
data  using  the  40-km  Eta  forecast  data  as  initialization  and  time-dependent  lateral  condition. 
These  statistical  parameters  were  obtained  using  10,528  data  during  15-day  period  in  April  2004. 

Table  3.  Statistical  parameters  between  the  MM5  forecast  and  surface  observation  data  using  the  40-km  Eta  as 
input  data. 


MD 

AD 

RMSE 

CC 

Temperature  (°C) 

1.6 

2.6 

3.4 

0.88 

Dew-point  temperature  (°C) 

-1.0 

3.0 

3.9 

0.54 

Wind  speed  (m/sec) 

0.9 

2.1 

2.8 

0.31 

Wind  vector  x-component  (m/sec) 

-0.1 

1.9 

2.9 

0.41 

Wind  vector  y-component  (m/sec) 

0.6 

2.2 

2.9 

0.54 

NOTE:  RMSVE  =  4. 1  m/sec,  MWDDF  =  46° 

The  statistical  parameters  obtained  using  the  GFS  data  as  input  data  are  shown  in  table  4;  the 
data  were  obtained  for  the  same  period  as  the  40-km  Eta  data. 
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Table  4.  Same  as  table  3,  except  for  the  GFS  as  input  data. 


MD 

AD 

RMSE 

CC 

Temperature  (°C) 

1.6 

3.1 

4.1 

0.82 

Dew-point  temperature  (°C) 

-0.8 

3.1 

4.0 

0.46 

Wind  speed  (m/sec) 

1.4 

2.4 

3.1 

0.26 

Wind  vector  x-component  (m/sec) 

-0.2 

2.2 

3.2 

0.38 

Wind  vector  y-component  (m/sec) 

0.0 

2.5 

3.3 

0.46 

NOTE:  RMSVE  =  4.52  m/sec,  MWDDF  =  45.4° 


Figures  15  through  18  are  time  series  of  the  statistical  parameters  (MD,  CC,  and  AD).  In  these 
figures,  different  colors — pink  for  the  40-km  Eta,  and  black  for  the  GFS — are  used  to  plot  the 
data. 


Figure  15.  Time  series  of  statistical  parameters  (MD,  CC,  and  AD)  for  temperature  between 
the  MM5  forecast  and  surface  observation  data. 
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Figure  16.  Same  as  figure  15,  except  for  dew-point  temperature. 


Figure  17.  Same  as  figure  15,  except  for  wind  speed. 
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RMSVE ( M/S )  &  MWDDFIDEG. ) 


Figure  18.  Time  series  of  RMSVE  and  MWDDF. 

As  seen  in  tables  3  and  4  and  figures  15  through  18,  the  MM5  produces  statistically  better 
forecasting  results  for  all  surface  meteorological  parameters  when  using  the  initialization  and 
lateral  boundary  data  from  the  40-km  Eta  rather  than  using  the  data  from  the  GFS. 

In  a  previous  study  (Henmi,  2003b),  for  the  same  model  domain  during  1 1-day  period  in 
February  2003,  the  MM5  produced  statistically  better  forecasting  results  for  all  surface 
meteorological  parameters  when  it  used  the  initialization  and  lateral  boundary  data  from  the  GFS 
rather  than  the  data  from  the  NOGAPS. 

Thus,  it  is  important  to  choose  appropriate  forecast  data  from  a  large-scale  forecast  model  as 
input  to  mesoscale  models  such  as  the  MM5  and  WRF. 

4.3  Study  of  the  WRF  Forecasting  Over  the  Domain  of  the  Western  Texas 

The  terrain  contour  plot  of  the  western  Texas  used  for  the  WRF  forecast  calculation  is  shown  in 
figure  4.  The  area  can  be  characterized  by  a  gradual  sloping  from  west  to  east,  and  it  is  flatter 
and  simpler  than  the  terrain  in  the  Utah  domain  (shown  in  fig.  3).  It  should  be  also  noted  that  in 
this  model  domain,  the  surface  observation  sites  were  distributed  much  more  evenly  than  those 
in  the  Utah  domain. 

The  statistical  results  given  in  this  section  were  based  on  about  18,000  data  points  from  twenty 
24-h  forecast  calculations  during  April  2004. 

The  statistical  parameters  (MD,  AD,  RMSE,  and  CC),  contrasted  between  the  WRF  forecast  and 
observation  data,  are  given  in  table  5. 
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Table  5.  Statistical  parameters  between  the  WRF  forecast  and  observation  data  over  the  western  Texas  domain. 


MD 

AD 

RMSE 

CC 

Temperature  (°C) 

2.5 

2.7 

3.7 

0.91 

Dew  Point  Temperature  (°C) 

2.9 

3.2 

4.5 

0.77 

Wind  Speed  (m/sec) 

-1.2 

1.9 

2.6 

0.49 

Wind  Vector  x-component  (m/sec) 

0.0 

1.8 

2.5 

0.72 

Wind  Vector  y-component  (m/sec) 

0.7 

2.2 

3.2 

0.78 

NOTE:  RMSVE  =  4. 1  (m/sec),  MWDDF  =  33° 

Scatter  diagrams  between  forecast  and  observation  data  for  temperature,  dew-point  temperature, 
and  wind  speed  are  displayed  in  figures  19,  20,  and  21,  respectively. 


Figure  19.  Scatter  diagram  between  the  WRF  forecast 
and  observation  data  over  the  western  Texas, 
for  temperature. 
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Figure  20.  Same  as  figure  19,  except  for  dew-point 
temperature. 


Figure  21.  Same  as  figure  19,  except  for  wind  speed. 

The  significant  over-forecasting  of  temperatures,  seen  at  the  upper-left  part  of  figure  19,  is 
caused  by  the  failures  of  the  WRF  (and  the  40-km  Eta)  to  simulate  the  passage  of  a  cold  front 
over  the  area  on  April  10,  2004. 

Time  series  of  the  statistical  parameters  are  shown  in  figures  22-25. 

In  figure  23,  which  shows  the  dew-point  temperature  statistics,  a  significantly  small  CC  and 
large  MD  and  AD  are  seen  at  the  early  hours  of  forecast,  with  peaks  at  0100  UTC.  Examination 
of  data  reveals  that  these  large  deviations  are  caused  by  large  values  of  dew-point  temperature 
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for  several  of  the  forecast  days.  In  this  study,  dew-point  temperature  at  the  2-m  level  is 
calculated  as  follows,  using  Tetan’s  formula  (Yamada,  1989): 


P2  =  A 


A  =  log 


fftV 

\T 2; 

p>*Q2 

3.8002 

V  J 


td2 


35.86  x  A  - 17.27  x  273.16 
A  - 17.27 


where 

p  :  pressure  at  2-m  level  (mbar), 

PQ :  standard  pressure  (1000  mbar), 

Q  :  potential  temperature  at  2-m  level  (°K), 
f2 :  temperature  at  2-m  level  (°K), 

Q  :  mixing  ratio  at  2-m  level  (g/kg),  and 
TD2  :  dew-point  temperature  (°K). 


K  = 


where 


(1) 

(2) 

(3) 


(4) 


Pd  :  gas  constant  for  dry  air 

(J  :  specific  heat  at  constant  pressure 

The  WRF  output  file  contains  Q  ,  p 2 ,  and  Q  .  Further  study  is  needed  to  examine  which 
parameter(s)  caused  the  large  dew-point  temperature  deviations  from  the  observation  data. 
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Figure  22.  Time  series  of  statistical  parameters  (MD,  CC,  and  AD)  between  the  WRF  forecast 
and  observation  data  over  the  western  Texas  domain  for  temperature. 


Figure  23.  Same  as  figure  22,  except  for  dew-point  temperature. 
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Figure  24.  Same  as  figure  22,  except  for  wind  speed. 


RMSVEIM/S)  &  MWDDF ( DEG . ) 
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Figure  25.  Time  series  of  RMSVE  and  MWDDF  for  the  WRF  forecast  and  observation  data 
over  the  western  Texas. 

The  tables,  scatter  diagrams,  and  time  series  of  the  statistical  parameters  reveal  that  the  WRF 
over-forecasted  temperature  and  dew-point  temperature  over  the  western  Texas  domain  during 
the  period  of  April  2004.  Similarly,  the  WRF  overestimated  temperature  and  dew-point 
temperature  over  the  Utah  domain.  Thus,  it  is  necessary  to  examine  whether  or  not  the  over¬ 
forecasting  of  temperature  and  dew-point  temperature  is  due  to  the  40-km  Eta  forecast  data  used 
as  initialization  and  time-dependent  lateral  boundary  condition. 
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Let  us  compare  the  statistics  of  the  WRF  forecast  over  the  Utah  domain  and  the  western  Texas 
domain,  although  such  a  comparison  is  qualitative: 

•  The  statistical  parameters  over  the  western  Texas  domain  are  superior  to  those  over  the 
Utah  domain  (see  tabs.  2  and  5). 

•  From  the  comparisons  of  scatter  diagrams  (figs.  6  and  19,  figs.  8  and  20,  and  figs.  10  and 
21),  the  correlations  between  the  WRF  forecast  and  observation  are  better  over  the  western 
Texas  domain  than  over  the  Utah  domain. 

The  better  statistical  results  may  have  been  a  result  of  the  simpler  terrain  features  and  better 
quality-controlled  observation  data  over  the  western  Texas  domain. 


5.  Conclusions 


The  forecasting  skills  of  surface  meteorological  parameters  of  two  mesoscale  models,  MM5  and 
WRF,  were  statistically  evaluated  over  two  different  geographical  areas,  Utah  and  western 
Texas.  Both  areas  have  networks  for  conducting  surface  meteorological  observation. 

A  triple-nested  MM5,  with  grid  resolutions  of  45,  15,  and  5  km  for  domains  1,  2,  and  3, 
respectively,  was  used  over  the  Utah  area.  For  the  present  study,  only  the  forecast  fields  for 
domain  3  were  compared  with  the  observed  data. 

The  WRF  version  1.3,  which  does  not  have  multiple-nesting  capability,  was  used  with  a  52-by- 
52  grid  size  and  a  5-km  grid  resolution.  The  model  was  applied  to  the  model  domains  of  both 
Utah  and  western  Texas.  Over  the  Utah  domain,  the  WRF  domain  covered  an  area  similar  to  the 
MM5,  domain  3. 

Using  the  45-km  Eta  forecast  data  as  input  data,  forecast  calculations  of  both  the  MM5  and  the 
WRF  were  carried  out,  and  the  results  were  compared  with  surface  observation  data.  Both 
models  tended  to  over-forecast  temperature  and  dew-point  temperature,  although  the  correlation 
coefficients  between  forecast  and  observation  were  fairly  high. 

The  statistical  parameters  for  MM5  were  slightly  better  than  those  for  the  WRF.  For  both  the 
MM5  and  the  WRF,  the  statistical  parameters  for  wind  vector  components  were  inferior  to  those 
of  temperature  and  dew-point  temperature,  although  the  WRF  values  were  slightly  better  than  the 
MM5  values. 

It  is  important  to  do  comparative  studies  of  the  MM5  and  WRF  with  similar  model  domain 
configurations.  With  recent  arrival  of  the  WRF  version  2,  which  has  a  multiple  nesting 
capability,  such  studies  have  now  become  possible. 

Also  studied  were  the  influences  of  different  input  data  for  initialization  and  time-dependent 
lateral  condition  on  the  MM5  forecast  fields.  To  do  this,  data  from  the  45-km  Eta  and  the  GFS 
were  used  and  the  results  compared.  For  all  surface  meteorological  parameters,  the  MM5  using 
the  45-km  Eta  perfonned  better  than  the  MM5  using  the  GFS.  The  importance  of  input  data  to 
mesoscale  models  should  be  further  studied  using  other  large-scale  model  outputs,  such  as  a  GFS 
with  0.5°  grid  resolution  and  a  45-km  Eta. 
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Also,  the  WRF  forecasting  over  the  western  Texas  domain  produced  better  statistical  results  than 
forecasting  over  the  Utah  domain,  probably  due  to  simpler  and  more  monotonous  terrain  found 
in  the  western  Texas  as  compared  to  Utah.  Furthermore,  the  surface  observation  data  network 
over  the  western  Texas  appears  to  provide  more  appropriate  datasets  for  mesoscale  model 
evaluation.  In  the  future,  this  Mesonet  dataset  will  be  used  for  mesoscale  model  evaluation  study. 
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Acronyms 


AD 

absolute  difference 

AVN 

Aviation  Forecast  Model 

BFM 

Battlescale  Forecast  Model 

CC 

correlation  coefficient 

GFS 

Global  Forecast  System 

MD 

mean  difference 

MM5 

Mesoscale  Model  Version  5 

MRF 

Medium  Range  Forecast  model 

MWDDF 

mean  wind  direction  difference 

NCAR 

National  Center  for  Atmospheric  Research 

NCEP 

National  Center  for  Environmental  Prediction 

NOGAPS 

Navy  Operational  Global  Atmospheric  Prediction  System 

PBL 

planetary  boundary  layer 

RMSE 

root  mean  square  error 

RMSVE 

root  mean  square  vector  error 

RRTM 

rapid  radiative  transfer  model 

TTU 

Texas  Tech  University 

UTC 

universal  time  coordinated 

WRF 

Weather  Research  and  Forecast  model 
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